Added new capability in ice_forcing.F90 to test with JRA55 atmospheri… #350

rallard77 · 2019-08-14T19:23:39Z

…c forcing on gx1 grid.

For detailed information about submitting Pull Requests (PRs) to the CICE-Consortium,
please refer to: https://github.com/CICE-Consortium/About-Us/wiki/Resource-Index#information-for-developers

PR checklist

Short (1 sentence) summary of your PR:
Added new capability to perform tests with JRA55 atmospheric forcing on gx1 grid for years 2005-2009.
Developer(s):
Richard Allard, Julie Crout, Matt Turner
Suggest PR reviewers from list in the column to the right.
Please copy the PR test results link or provide a summary of testing completed below.
repo = https://github.com/rallard77/CICE.git
#bran = jra55
#hash = eff67a2
#hshs = eff67a2
#hshu = Elizabeth Hunke [email protected]
#hshd = Thu Aug 1 17:36:22 2019 -0600
#date = 2019-08-13
#time = 14:28:39
#mach = conrad
#user = allard
#vers = CICE_6.0.1
...skipping most of the log file
PASS conrad_gnu_restart_gx1_8x1_jra55 build
PASS conrad_gnu_restart_gx1_8x1_jra55 initialrun
PASS conrad_gnu_restart_gx1_8x1_jra55 run 94.60 16.34 47.64
PASS conrad_gnu_restart_gx1_8x1_jra55 test
#---
COPY conrad_gnu_smoke_gx3_1x1_diag1_run1day build
PASS conrad_gnu_smoke_gx3_1x1_diag1_run1day run 9.00 1.52 4.49
PASS conrad_gnu_smoke_gx3_1x1_diag1_run1day test
#---
COPY conrad_gnu_restart_gbox128_8x1_diag1 build
PASS conrad_gnu_restart_gbox128_8x1_diag1 initialrun
PASS conrad_gnu_restart_gbox128_8x1_diag1 run 30.07 8.70 15.00
PASS conrad_gnu_restart_gbox128_8x1_diag1 test
#---
COPY conrad_gnu_restart_gx3_4x2_debug_diag1_run5day build
PASS conrad_gnu_restart_gx3_4x2_debug_diag1_run5day initialrun
PASS conrad_gnu_restart_gx3_4x2_debug_diag1_run5day run 50.30 22.29 14.04
PASS conrad_gnu_restart_gx3_4x2_debug_diag1_run5day test
#-------

#totl = 708
#pass = 708
#fail = 0
#pend = 0

ENTER INFORMATION HERE

…c forcing on gx1 grid.

apcraig

This looks good. Just a few questions.

It looks like a test was removed from the base suite, was this intended?

The "if" section around line 1380 of ice_forcing.F90 looks a little odd. Is this correct mixing logic for atm_data_type and atm_data_format?

  if (trim(atm_data_type) == 'hadgem') then ! netcdf
     i = index(data_file,'.nc') - 5
     tmpname = data_file
     write(data_file,'(a,i4.4,a)') tmpname(1:i), yr, '.nc'
  elseif (trim(atm_data_format) == 'nc') then ! netcdf
     i = index(data_file,'.nc') - 5
     tmpname = data_file
     write(data_file,'(a,i4.4,a)') tmpname(1:i), yr, '.nc'
  else                                     ! LANL/NCAR naming convention
     i = index(data_file,'.dat') - 5
     tmpname = data_file
     write(data_file,'(a,i4.4,a)') tmpname(1:i), yr, '.dat'
  endif

Have the new data files been put on ncar ftp so others can test? Have the data files been tested from a formal CICE input data area? I don't think I see the input files on conrad in the shared input data area.

What is the plan for updating documentation. Could this be done with this PR?

dabail10 · 2019-08-21T19:59:14Z

I'm working on this now. I am in the process of setting up the data tar files on our ftp. I am trying a test on cheyenne for a year. One quirky thing I have noticed is that it is ignoring year_init/fyear_init and starting from 2025 when I use year_init = 2005. I assume this is because it is using the istep1 and time from the initial file which is year Jan 1st year 21. Shouldn't the code be ignoring this on an initial run?

dabail10 · 2019-08-21T20:13:33Z

Just found the use_restart_time flag.

dabail10 · 2019-08-21T20:18:13Z

Dumb question. Does year_init have to line up with fyear_init to read in the forcing?

dabail10

The code changes are working fine here. I would recommend adding the namelist flag:

use_restart_time = .false.

to the test. I am working on setting up the forcing tar files. This will be required if we add the JRA55 set_nml option as a test in Travis-CI.

apcraig · 2019-08-22T15:50:41Z

@rallard77, looks like there are a couple things pending then we can possibly merge this.

review the changes in base_suite.ts, it looks like a test was removed when you added your test. Can you fix that.
please add "use_restart_time = .false." in the set_nml.jra55 file
has the documentation been adequately updated?

While that is going on, I will try to independently download the data from the ftp site and run a test on another machine just to make sure the data is available. Thanks.

dabail10 · 2019-08-22T17:56:40Z

I have updated all of the datasets on our ftp to include the JRA55 forcing. Note that these are really big and the file sizes have increased dramatically.

CICE_data_all.tar.gz
CICE_data_new.tar.gz
CICE_data_forcing_gx1_200?.tar.gz

Also, should we put use_restart_time = .false. in the default ice_in? Does this impact continue runs?

Dave

apcraig · 2019-08-22T18:00:43Z

I don't know much about the use_restart_time flag. Did you need it to get the case to run? By default, it's true and we do not have any tests that change it to false, so we are not exercising it.

dabail10 · 2019-08-22T18:02:12Z

It runs fine. Just that the date of the history and restart files are off by 20 years from year_init.

Dave

apcraig · 2019-08-22T19:54:50Z

I downloaded the "new" CICE tar file to conrad, dropped it in the inputdata area, and ran conrad_intel_restart_gx1_16x2_jra55 successfully.

I guess I think we should do a bit more research on the "use_restart_time" flag before switching it for this case. So I guess I feel there are still two minor questions before merging @rallard77

add the removed test in base_suite.ts if appropriate
let us know whether the documentation is adequate at this point and if more is needed whether you propose to defer.

eclare108213

This looks good, though I agree that a few additional things need to be done before the PR is merged:

Definitely reinstate the bgc test in the base suite.
We do need more documentation about this case, to at least describe what the forcing fields are, what dates they cover, and a citation for the original data set. This can be added in a separate PR but I'd prefer it all go in together.

The calendar related flags in CICE were made to be very flexible, perhaps too flexible -- this is something that can be changed when the time manager is overhauled. Generally speaking, year_init and fyear do not need to be the same, but in this case I think they should be the same. If use_restart_time is true, then I believe it overrides the other flags, except possibly the istep0 value (which I think gets added to whatever the others produce), so it should be false. Note that ycycle=1 indicates that only 1 year of forcing data is being used.

What appears to be going on here is that the new test option mimics the set_nml.gx1 file, which is basically just a smoke test with 24 time steps, and the forcing and simulation dates might not match -- something we hadn't noticed. A full 1- or 5-year test with this data would need additional namelist flags as in set_nml.gx1prod and/or set_nml.qc (but with values appropriate for this data set). It's probably better to set everything appropriately in the set_nml.jra55 file except keep the run short for a smoke test.

I suggest setting the flags so that the dates on the history and restart files match the dates inherent to the data for longer runs.

This should probably be done for the COREII forcing too, but not in this PR. At some point when we're comfortable with the JRA55 data set, maybe we can switch the QC to this data and retire the COREII data or at least separate it out from the JRA55 download. This will require some longer simulations to fully test and add sample output.

rallard77 · 2019-08-22T22:45:59Z

All, I am at IGS, Let me double check on Monday. Rick

…

On 8/22/2019 1:02 PM, David A. Bailey wrote: It runs fine. Just that the date of the history and restart files are off by 20 years from year_init. Dave — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#350?email_source=notifications&email_token=AG63LB5BB54CCYHZPJZUHXTQF3ICLA5CNFSM4ILYD6L2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD455AXQ#issuecomment-524013662>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AG63LB4RAJDB36E3DQXIZK3QF3ICLANCNFSM4ILYD6LQ>.

-- logo *Richard A. Allard * *Head, Nearshore and Coupled Models Systems Section* *Oceanography Division NRL Code 7322* *Stennis Space Center, MS 39529* *T: 228.688.4894* [email protected] <mailto:[email protected]> **** ******

apcraig · 2019-08-22T22:49:19Z

I support the idea that the setup should be correct for a long run even if we are just carrying out short test runs.

rallard77 · 2019-08-22T22:54:15Z

I provided some additional documentation to "ug_case_settings.rst" but could easily add more details and a description of JRA55. Where should I put that? Rick

…

On 8/22/2019 2:54 PM, Tony Craig wrote: I downloaded the "new" CICE tar file to conrad, dropped it in the inputdata area, and ran conrad_intel_restart_gx1_16x2_jra55 successfully. I guess I think we should do a bit more research on the "use_restart_time" flag before switching it for this case. So I guess I feel there are still two minor questions before merging @rallard77 <https://github.com/rallard77> * add the removed test in base_suite.ts if appropriate * let us know whether the documentation is adequate at this point and if more is needed whether you propose to defer. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#350?email_source=notifications&email_token=AG63LB3EO25PEICLBF2H4NLQF3VIXA5CNFSM4ILYD6L2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD46GUEY#issuecomment-524053011>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AG63LB4SULHVJTWCKEQ334LQF3VIXANCNFSM4ILYD6LQ>.

-- logo *Richard A. Allard * *Head, Nearshore and Coupled Models Systems Section* *Oceanography Division NRL Code 7322* *Stennis Space Center, MS 39529* *T: 228.688.4894* [email protected] <mailto:[email protected]> **** ******

duvivier · 2019-08-26T22:39:27Z

I just emailed @rallard77 but we also need to update the CICE wiki with information about the new forcing data:
https://github.com/CICE-Consortium/CICE/wiki/CICE-Input-Data

Mainly the "Updated Files" section as well as the directory structures and sizes. @dabail10 do you want to handle this since you put them together or should I?

dabail10 · 2019-08-27T03:03:34Z

I can update the wiki.

dabail10 · 2019-08-27T19:32:57Z

Updated the forcing wiki. Let me know if you expected more than this?

rallard77 · 2019-08-28T16:23:46Z

All, With use_restart_time set to false, year_init and fyear_init set to 2005 (could be any year between 205-2009), the history and restart files all have dates with 2005 embedded in the file name. I beleive this is what we want included in the test. One can run a full year with these settings with a modification to npt. Rick

…

On 8/22/2019 5:38 PM, Elizabeth Hunke wrote: ***@***.**** requested changes on this pull request. This looks good, though I agree that a few additional things need to be done before the PR is merged: * Definitely reinstate the bgc test in the base suite. * We do need more documentation about this case, to at least describe what the forcing fields are, what dates they cover, and a citation for the original data set. This can be added in a separate PR but I'd prefer it all go in together. The calendar related flags in CICE were made to be very flexible, perhaps too flexible -- this is something that can be changed when the time manager is overhauled. Generally speaking, year_init and fyear do not need to be the same, but in this case I think they should be the same. If use_restart_time is true, then I believe it overrides the other flags, except possibly the istep0 value (which I think gets added to whatever the others produce), so it should be false. Note that ycycle=1 indicates that only 1 year of forcing data is being used. What appears to be going on here is that the new test option mimics the set_nml.gx1 file, which is basically just a smoke test with 24 time steps, and the forcing and simulation dates might not match -- something we hadn't noticed. A full 1- or 5-year test with this data would need additional namelist flags as in set_nml.gx1prod and/or set_nml.qc (but with values appropriate for this data set). It's probably better to set everything appropriately in the set_nml.jra55 file except keep the run short for a smoke test. * I suggest setting the flags so that the dates on the history and restart files match the dates inherent to the data for longer runs. This should probably be done for the COREII forcing too, but not in this PR. At some point when we're comfortable with the JRA55 data set, maybe we can switch the QC to this data and retire the COREII data or at least separate it out from the JRA55 download. This will require some longer simulations to fully test and add sample output. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#350?email_source=notifications&email_token=AG63LBZ2ZMEUDZG4OUXO7SDQF4IOXA5CNFSM4ILYD6L2YY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCCOPMAQ#pullrequestreview-278722050>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AG63LBYSZQVAE2JZLBYQEQLQF4IOXANCNFSM4ILYD6LQ>.

apcraig · 2019-09-16T22:37:59Z

Thanks @mattdturner. I guess I suggest that you create a new option, set_nml.jra55_2008, for the 2008 start date leap year test. I would then setup a test that was "smoke, jra55_2008, 3 months, medium"? That way we clear March 1 for sure. I think this is really mostly a test of the leap year capability with the JRA dataset?

With the current test, "smoke gx1 8x1 jra55,short", this is not using the leap year calendar, is that correct? I would modify this to be a restart test and I would increase the pe count (you are running gx1) to something 10x-32x higher? Does that make sense? I haven't quite followed all the details of all the discussions, but my sense is we probably want a restart test here and to be able to run qc if needed. Then separately, we want to have a leap year test with the leap year calendar.

Are we thinking that eventually we'll drop the COREII dataset testing? Or are we going to continue to support both? If JRA is going to be our primary forcing dataset, we might want to add bgc gx1 tests with JRA forcing too.

One final issue I'm noticing, the set_nml.gx1 sets the COREII forcing dataset then set_nml.jra55 switches that to JRA55. Ideally, we'd clean this up a bit, maybe renaming gx1 to gx1_coreii and adding gx1_jra55. or maybe removing the coreii stuff from set_nml.gx1 and then adding a set_nml.coreii. But I think we can defer this for the time being for backwards compatibility. I assume you've confirmed that by using gx1 and jra55, the tests are being setup with the jra forcing correctly.

…lization

…st option file

…so add a restart test for JRA55 in base_suite

mattdturner · 2019-09-17T11:35:51Z

I guess I suggest that you create a new option, set_nml.jra55_2008, for the 2008 start date leap year test. I would then setup a test that was "smoke, jra55_2008, 3 months, medium"? That way we clear March 1 for sure. I think this is really mostly a test of the leap year capability with the JRA dataset?

Correct, the smoke test is basically just to test the use_leap_year option with the JRA55 dataset.

With the current test, "smoke gx1 8x1 jra55,short", this is not using the leap year calendar, is that correct?

Yes

Are we thinking that eventually we'll drop the COREII dataset testing? Or are we going to continue to support both?

My understanding is that, eventually, JRA55 would replace COREII. I could be wrong though.

I've pushed changes that add a new jra55_2008 namelist option, and a new run90day option. It also changes the tests in base_suite.ts to do a 90 day smoke test for jra55_2008, and a JRA55 smoke test.

I am currently running base_suite to make sure I didn't break anything, and will post the results once all simulations finish.

mattdturner · 2019-09-17T14:47:56Z

So far, all tests have passed. There are still 2 tests that I am waiting for (conrad_intel_restart_gx1_40x4_droundrobin_medium and conrad_intel_restart_tx1_40x4_dsectrobin_medium), but everything else has completed and passed. Since those 2 tests shouldn't be affected by the changes that were made any differently than the other tests, I'm comfortable saying that there will be no failures in the base suite.

I don't think any documentation needs to be updated for the recent changes I made. I have no idea if the JRA55 documentation is sufficient (I'm not suggesting that it isn't, either). Somebody more familiar with JRA55 would need to make that conclusion.

duvivier · 2019-09-18T19:00:31Z

doc/source/user_guide/ug_case_settings.rst

@@ -338,6 +338,7 @@ Table of namelist options
   "","``restore_ice``", "true/false", "restore ice state along lateral boundaries", ""
   "\*","``atm_data_type``", "``default``", "constant values defined in the code", ""
   "","", "``LYq``", "AOMIP/Large-Yeager forcing data", ""
+   "","", "``JRA55``", "JRA55 forcing data", ""


Let's add the citation here. You want to do it with the following format :cite:Tsujino18 (see below for box2001 as an example).

You will also have to add the citation at the very en of the following file (right after Roberts18: /doc/source/master_list.bib

@Article{Tsujino18,
author = "H. Tsujino and S. Urakawa and R.J. Small and W.M. Kim and S.G. Yeager and et al.",
title = "{JRA‐55 based surface dataset for driving ocean–sea‐ice models (JRA55‐do)}",
journal = OM,
year = {2018},
url = {http://dx.doi.org/10.1016/j.ocemod.2018.07.002}
}

I added the citation.

duvivier

See comments in ug_case_settings.rst

duvivier · 2019-09-18T19:46:08Z

@rallard77 @apcraig @eclare108213 @dabail10

I have gone through the CICE Wiki (https://github.com/CICE-Consortium/CICE/wiki/CICE-Input-Data) to add the JRA55 information in the Input Data section. Please check that the citations are correct for JRA55 and COREII.

Additionally, I noticed that the yearly forcing didn't include JRA55, so I'm remaking the forcing files and will put them in the FTP asap. I also removed the other "new" files since they were from a year ago and we say on the wiki that after 6 months we no longer consider the data to be "new".

Otherwise I've worked with @rallard77 on the documentation through the repo and minus a few small details we should be ready to go here soon.

duvivier · 2019-09-19T02:57:42Z

I just updated the wiki pages with the individual year files and sizes of files. I think we're all good to go on that end of the documentation. Let me know if anyone sees an issue there.

So just the addition of the citation into the rst files in the main repo.

mattdturner · 2019-09-19T13:22:00Z

So just the addition of the citation into the rst files in the main repo.

The citations should have been added to the rst files already. So it sounds like this PR might be good to go

duvivier · 2019-09-19T13:24:55Z

So just the addition of the citation into the rst files in the main repo.

The citations should have been added to the rst files already. So it sounds like this PR might be good to go

@mattdturner Yep, I see them in there now. Looks good to me.

apcraig

I just had another look at the code modifications and I don't have any concerns at this point. Unless there are outstanding issues still to be done that I'm not aware of or @eclare108213 has some comments, I think we can merge.

eclare108213 · 2019-09-19T16:10:23Z

This looks fine to me except for the conflicting file.
Not for this PR but for the release, can one of you who has done a run with this data set (preferably using a standard test from the test suite) make some plots similar to those with COREII, for the sample output wiki page? Thanks everyone! This is a big step for us.

duvivier · 2019-09-19T16:49:09Z

@mattdturner it looks like the conflict is with the updated documentation file. This is one I changed yesterday to add the reference for COREII. I think if you just do a "git pull upstream master" to the development fork (assuming the master branch is your upstream) that will take care of it.

apcraig · 2019-09-19T16:58:04Z

I just edited the conflicted doc file in the PR. Sometimes you can do that with a conflict. I committed the change thru the github interface. That correction is now being checked with travis and readthedocs. Once those both pass, we should be able to merge.

duvivier · 2019-09-19T17:03:08Z

@apcraig It looks like the documentation build is good, so as soon as Travis passes I think we can merge.

rallard77 · 2019-09-19T23:13:19Z

Sure, I will be able to put together some plots.... Rick

…

On 9/19/2019 11:10 AM, Elizabeth Hunke wrote: This looks fine to me except for the conflicting file. Not for this PR but for the release, can one of you who has done a run with this data set (preferably using a standard test from the test suite) make some plots similar to those with COREII, for the sample output wiki page? Thanks everyone! This is a big step for us. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#350?email_source=notifications&email_token=AG63LBZCOXLEFGL53EIU3ALQKOP7BA5CNFSM4ILYD6L2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7EAJ7Q#issuecomment-533202174>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AG63LBYLZDKPTX32THSR5ETQKOP7BANCNFSM4ILYD6LQ>.

rallard77 · 2019-09-20T18:59:10Z

All, Attached are a few slides for consideration for the wiki. The first is taken from the wiki and slides 2 and 3 are my suggested contributions. We ran a full year using JRA55 with gx1 for 2008. The actual test in the base_suite runs for 60 days in 2008 through March 1. The users should replicate the March 1 results. If they run for a full year, they should also get the same results for Sept 1. I can change these slides based on feedback. Rick

…

On 9/19/2019 11:10 AM, Elizabeth Hunke wrote: This looks fine to me except for the conflicting file. Not for this PR but for the release, can one of you who has done a run with this data set (preferably using a standard test from the test suite) make some plots similar to those with COREII, for the sample output wiki page? Thanks everyone! This is a big step for us. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#350?email_source=notifications&email_token=AG63LBZCOXLEFGL53EIU3ALQKOP7BA5CNFSM4ILYD6L2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7EAJ7Q#issuecomment-533202174>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AG63LBYLZDKPTX32THSR5ETQKOP7BANCNFSM4ILYD6LQ>.

duvivier · 2019-09-20T19:12:58Z

@rallard77 I don't see attached plots. Maybe email them directly to me?

eclare108213 · 2019-09-20T19:51:02Z

Or drag and drop into a comment box directly on this github page.

Added new capability in ice_forcing.F90 to test with JRA55 atmospheri…

2cabe04

…c forcing on gx1 grid.

eclare108213 added Forcing Priority: High Testing Type: Feature labels Aug 14, 2019

eclare108213 self-assigned this Aug 14, 2019

apcraig reviewed Aug 14, 2019

View reviewed changes

apcraig requested review from dabail10, duvivier and eclare108213 August 14, 2019 23:31

Added JRA55 as an option under atm_data_type

1665fbb

dabail10 approved these changes Aug 21, 2019

View reviewed changes

eclare108213 requested changes Aug 22, 2019

View reviewed changes

phil-blain mentioned this pull request Aug 28, 2019

Problem with 'use_leap_years' #343

Closed

turner added 5 commits September 17, 2019 11:28

fix a bug in the newly-implemented use_leap_years check during initia…

5324441

…lization

add use_leap_years and use_restart_time specification to jra55 nameli…

91e0a63

…st option file

add new jra55_2008 and run90day namelist option files

a2d7a78

modify the JRA55 smoke test to be a 90 day test and start in 2008. Al…

f01304e

…so add a restart test for JRA55 in base_suite

Merge branch 'jra55' of https://github.com/rallard77/CICE into jra55

85814ad

duvivier reviewed Sep 18, 2019

View reviewed changes

duvivier requested changes Sep 18, 2019

View reviewed changes

update documentation to include citation for JRA55

81db094

duvivier approved these changes Sep 19, 2019

View reviewed changes

apcraig approved these changes Sep 19, 2019

View reviewed changes

eclare108213 approved these changes Sep 19, 2019

View reviewed changes

Merge branch 'master' into jra55

b618fcd

apcraig merged commit 3d1a340 into CICE-Consortium:master Sep 19, 2019

eclare108213 mentioned this pull request May 20, 2021

JRA forcing initialization/spinup #481

Closed

Added new capability in ice_forcing.F90 to test with JRA55 atmospheri… #350

Added new capability in ice_forcing.F90 to test with JRA55 atmospheri… #350

Conversation

rallard77 commented Aug 14, 2019 • edited by eclare108213 Loading

PR checklist

apcraig left a comment

Choose a reason for hiding this comment

dabail10 commented Aug 21, 2019

dabail10 commented Aug 21, 2019

dabail10 commented Aug 21, 2019

dabail10 left a comment

Choose a reason for hiding this comment

apcraig commented Aug 22, 2019

dabail10 commented Aug 22, 2019

apcraig commented Aug 22, 2019

dabail10 commented Aug 22, 2019

apcraig commented Aug 22, 2019

eclare108213 left a comment

Choose a reason for hiding this comment

rallard77 commented Aug 22, 2019 via email

apcraig commented Aug 22, 2019

rallard77 commented Aug 22, 2019 via email

duvivier commented Aug 26, 2019

dabail10 commented Aug 27, 2019

dabail10 commented Aug 27, 2019

rallard77 commented Aug 28, 2019 via email

apcraig commented Sep 16, 2019

mattdturner commented Sep 17, 2019

mattdturner commented Sep 17, 2019

duvivier Sep 18, 2019

Choose a reason for hiding this comment

mattdturner Sep 18, 2019

Choose a reason for hiding this comment

duvivier left a comment

Choose a reason for hiding this comment

duvivier commented Sep 18, 2019

duvivier commented Sep 19, 2019

mattdturner commented Sep 19, 2019

duvivier commented Sep 19, 2019

apcraig left a comment

Choose a reason for hiding this comment

eclare108213 commented Sep 19, 2019

duvivier commented Sep 19, 2019

apcraig commented Sep 19, 2019

duvivier commented Sep 19, 2019

rallard77 commented Sep 19, 2019 via email

rallard77 commented Sep 20, 2019 via email

duvivier commented Sep 20, 2019

eclare108213 commented Sep 20, 2019

rallard77 commented Aug 14, 2019 •

edited by eclare108213

Loading